Augmented reality (AR) is a technology that allows virtual objects to be mixed, e.g. in real time, with real world objects in an augmented reality scene that may be represented in one or more images displayed on a display device, such as a head-mounted display (HMD), a tablet, a Smartphone or a television display screen. This technology is also called mixed reality.
Interactions between the real and virtual objects of the AR scene may happen, for example when a virtual object moves in the AR scene and hits one or more real objects. For example, if a real scene is composed of a card box laid on a table and is augmented with a virtual sphere rolling on the table, it is expected that, when the sphere will collide the box, not only the sphere trajectory may be modified but also the box may move (from some slight oscillations to a strong projection depending on the weights and the velocities of the objects). Leão et al. discuss in “Geometric modifications applied to real elements in augmented reality” (2011 XIII Symposium on Virtual Reality) an AR system capable of performing modifications in real objects, in real time, aiming at providing realistic means of interaction among real and virtual objects.
FIG. 1A shows an example of an image 10 representing a real scene used in the AR system of Leão et al, the scene comprising real objects among which a real object 100. The real object 100 corresponds to a cube, e.g. a card box. A marker 101 is applied on a face of the real object 100, the marker 101 being used to get the camera position in the real world with regard to the real object, for proper insertion of virtual object. FIG. 1B shows an image 11 into which the real object 100 is replaced with a virtual replica 110 of the real object 100, a deformation being applied to the virtual replica 110. The virtual replica 110 is, for example, obtained by using a 3D model of the real object 100. The virtual replica 110 is textured by using a dynamic texture obtained from the image 10 of the real object 100. In the original scene 10, the top surface of the real object 100 is occluded by the real object 100 itself, and in the AR scene of the image 11, the upper face 112 is displayed, with an incorrect texture. The texture of the visible face of the real object 100 that comprises the marker 101 is retrieved from the image 10 and reused as texture information for the front face 111 and the upper face 112 of the virtual replica 110, leading to an issue regarding the realism of the image 11.